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LIFE CYCLE COSTING OF SIMULATED VS ACTUAL EQUIPMENT 
FOR INTERMEDIATE MAINTENANCE TRAINING 

F. Thomas Eggemeier and Gary A. Klein 

Logistics and Technical Training Division 
Air Force Human Resources Laboratory 
Wright-Patterson Air Force Base Ohio A5433 

Life cycle cost estimates of training equipment for F-16 Avionics Intermediate 

Station personnel were developed. The major purpose was to compare the cost 

of intermediate level maintenance training when conducted on simulated vs 

actual avionics test equipment. This was the initial phase of a planned 

two-part effort. The analysis was therefore limited to estimates of training 

device acquisition and maintenance costs. Total estimated fifteen-year costs 

for simulated equipment trainers were approximately 5056 less than comparable 

estimates for actual equipment trainers. 

INTRODUCTION 

This paper reports the initial results of a two-phase effort to develop 
life cycle cost (LCC) estimates of training equipment for F-16 Avionics 
Intermediate Station (AIS) maintenance personnel. The goal of the entire 
effort is to estimate the LCC of simulated test equipment versus actual test 
equipment in training personnel to operate and maintain the F-16 AIS. This 
initial phase was a preliminary analysis of major cost factors differentiating 
simulated and actual test equipment. It was conducted to provide an early 
estimate of the cost of a training simulator and also to decide if a more 
detailed LCC study was warranted. 





The AIS is the set of maintenance equipment required to test aircraft 
avionics units which have malfunctioned on the aircraft and have therefore 
been removed. The AIS is used to identify the specific fault in the unit so 
that the operator can remove and replace the subcomponent responsible for the 
malfunction. The operator is responsible for using the AIS to test, 
troubleshoot, and perform maintenance actions on units. Maintainer personnel, 
on the other hand, are responsible for maintaining and repairing the AIS 
itself. In this context, the AIS itself constitutes one training device which 
could be used to support the training of operators and maintainer personnel, 
and a simulated AIS would constitute an alternative training device. 

Simulators have a number of potential advantages over actual equipment in 
maintenance training, primarily in the areas of cost, safety, and reliability 
(Miller & Rockway, 1975; Modrick, Kanarick, Daniels, & Gardner, 1975). For 
example, since actual equipment trainers normally operate with high voltages, 
instructors are often reluctant to allow students to perform hands-on 
training, fearing injury to the student or damage to the equipment. 
Consequently, the effectiveness of the trainer is compromised. Several other 
factors contribute to the potential for increased effectiveness of simulators 
relative to actual equipment. These include the ability of the simulator to 
train the repair of a broader rangfe of malfunctions, permit greater 
environmental control, and provide a wider variety of instructional actions 
and conditions. Ease of device modification and greater reliability are also 
potential advantages of simulation over actual equipment. These potential 
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advantages make simulation an alternative worthy of serious consideration when 
acquiring training equipment. 

On the other hand, the use of actual AIS equipment for training has some 
potential advantages when compared to a simulated equipment trainer. The 
actual equipment is more realistic, thereby potentially promoting greater user 
acceptance. It is readily available as an add-on procurement. Simulators 
require special procurements, the developmeht of new hardware and software, 
and tne introduction of new maintenance requirements into the supply system. 

Among the potential differences between simulators and actual equipment, 
cost is obviously of major importance in deciding which type of training 
equipment to procure. This paper addresses the cost aspect of choice of a 
training device. 


METHOD 

Cost Model 

Cost factors for the analysis were selected from a training cost model 
developed by Braby, Henry, Parrish, and Swope (1975). The model includes the 
provision for costing a variety of training program factors. The model 
includes six major areas: facilities, equipment, instructional material, 
personnel, supplies, and students. 

The Braby et al. (1975) model has been previously used to estimate costs 
of intermediate maintenance simulators (Daniels & Cronin, 1975; Daniels, 
1976). Both previous efforts demonstrated a significant potential for cost 
savings with use of simulated equipment. Both studies also indicated that 
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equipment costs and school personnel costs were the two major areas which were 
significantly affected by use of simulators rather than actual equipment. In 
both studies, these two categories accounted for over 95* of the estimated LCC 
savings with use of simulated equipment. Of the two factors, equipment costs 
represented the most significant element differentiating actual from simulated 
equipment and accounted for more than 60* of the estimated savings. 

Since equipment costs represent the majority of estimated savings 
attributed to simulation in previous analyses, it was decided that this 
preliminary estimate would be conducted with primary focus on equipment 
costs. Emphasis in this effort was therefore placed on the cost factor with 
the greatest potential to show a clear differentiation between simulation and 
actual equipment trainers. 

Procedure 

A design team, consisting of training psychologists and a simulation 
engineer, was assembled to perform the analysis. To meet the objective of 
comparing simulated and actual equipment costs, the design team applied a 
three-step process. During Step One, analyses were conducted to determine a 
set of training requirements for test station operators and maintainers. Step 
Two consisted of developing a conceptual design for a representative AIS 
simulation that would satisfy those training requirements. During Step Three, 
acquisition and maintenance costs were derived for the training simulator and 
for an actual equipment trainer. 
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Derivation of Training Requirements 

Operator Training Requirements . Training requirements should, ideally, he 
derived from a complete Instructional Systems Development (ISD) analysis. 
However, ISD data were not available for use during this effort. Therefore, 
it was necessary to identify operator training requirements orimarily through 
an analysis of task data provided in an F-16 Task and Skill Analysis Report. 
Tnis report was supplemented with information gathered through interviews witn 
F-16 personnel and F-15 AIS instructor and maintenance personnel. A technioue 
of analyzing analogous training situations to derive the training requirements 
of new systems (Cream, Eqgemeier, & Klein, 1978) was used in this effort. 
Cnoice of the F-15 AIS was based on the fact that it was the closest existing 
analog to the planned F-16 AIS. 

An F-16 AIS consists of four test stations. Each station is automatic, in 
that its software is capable of performing the majority of necessary tests 
without operator intervention, once the aircraft avionics unit has been 
connected to the station. Avionics units are connected to the test station by 
means of interface test adaptors. The automatic testing procedures are 
designed to identify malfunctions to the subcomponent level. The subcomponent 
can then be removed and replaced by the operator. 

The task analyses and interview data indicated that in general, the 
primary functions of the F-16 AIS operator are to set up the test station, 
perform the necessary procedures to test the units, remove and replace 
defective subcomponents, and verify proper operation of the avionics units 
after they nad been repaired. 
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Task data were gathered for each avionics unit at each test staticni and 
summarized in a matrix format. From the matrix, it was possible to identify 
patterns of operator tasks and relate these to avionics units. The matrix was 
then used to determine the minimum number of units which would ensure training 
of each of the tasks required to operate the test station. On the basis of 
this procedure, twelve of thirty-four avionics units were identified as 
training requirements. Only the peripheral equipment and the interface test 
adaptors required to test these selected units were included as training 
requirements. 

Maintainer Training Requirements . Derivation of training requirements for 
maintainer personnel was more complex than for the operator, due to the 
absence of any maintainer task data. During interviews, F-15 AIS and F-16 
engineering prsonnel directly identified areas within the test station that 
would be expected to fail most frequently and be most difficult to repair. 
While this procedure was not as thorough as listing all tasks, it provided a 
direct assessment of the anticipated major training requirements. 

It was determined that the major requirement for the station maintainer is 
access to the interior of the test station in order to effect repairs. 
Interviews pinpointed the computational and switching element within a test 
station as the areas of least reliability. It was therefore decided that the 
repair of the Computer and the Switching Unit/Measurement and Stimulus System 
(SUMSS) elements in the F-16 AIS should be among the primary training 
requirements. 
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Development of Simulation Requirements 

The methods used to derive necessary simulation capability from training 
requirements have been described by Cream, Eggemeier, and Klein (1975, 1978). 
In the case of the operator tasks, the derivation was straightforward. A set 
of representative avionics units, test adaptors, and peripheral equipment had 
been identified as necessary to provide adequate training. The simulation 
requirements consisted of providing these elements of equipment and the 
components of the test station itself that were involved in testing avionics 
units. 

It was determined that simulation of only front panels on each test 
station was required for the operator, who typically works with only the 
exterior portions of the station. Three representations of front panels were 
used: (1) actual equipment; (2) simulated panels with knobs, switches, and 
displays functional; and (3) mock-up via photo or other type of flat 
representation. In order to provide a direct LCC comparison with actual 
equipment, a simulator configuration which included all four test stations was 
developed. 

Determination of simulator capabilities for maintainer personnel was also 
based upon the primary training requirements that had been identified. 

Training requirements were developed in terms of the electronic cards within 
the test station. The determination of simulator capabilities indicated that 
the trainee should be able to perform troubleshooting actions down to the 
level of testing cards and components of cards within the test station. 
Simulation of a core set of SUMSS cards and a number of computer sub-assembly 






card modules was therefore necessary to provide the maintainer with 
trouDleshooting experience on each category of card function within the 
station. 

The frequency of use of each of the SUMSS cards was determined from the 
F-16 AIS Preliminary Design Review data, as was the circuit function, 
connection in the test station, and distribution throughout the test station. 
Cards were then selected so that a representative of each major functional 
category was included in the simulation. On the basis of that criterion, 29 
cards distributed across the four test stations were chosen for simulation. 

The remaining 316 cards were designated for mock-up only. 

Pertinent literature was assembled and analyzed to determine computer 
Chassis access procedures, computer circuit functions, and computer card 
complexity. Card function was the criterion used to assess the appropriate 
degree of simulation for the computer. Subassembly computer card modules were 
chosen for simulation to represent a broad range of functions. On the basis 
of these criteria, ten computer card modules were chosen for some degree of 
simulation. Simulated operating components, simulated components, and photo 
or other flat representation mock-ups comprised the major simulation 
categories. 


RESULTS 

Simulator Acquisition Cost Estimate 

Estimates of acquisition costs for each simulated test station were 
developed. Acquisition cost estimates were based on three major cost 
factors: (1) Design and fabrication of simulators; (2) Engineering change 
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proposals during the simulator development cycle; and (3) Technical Order data 
required to support the simulators when they had been placed in the field. 

Design and fabrication costs included estimated design and software 
development costs, costs for fabrication of simulated test stations, and the 
cost of the materials required for the fabrication. Cost estimates in each 
case were based upon "best commercial practices." The total estimated design 
and fabrication cost for the simulated test stations was $3.3 million (M) 
dollars. 

In order to estimate the costs of engineering change proposals, it was 
assumed that a four-year development cycle would be required to procure the 
simulated stations. Engineering change costs were then estimated in the 
following manner: 20% of the estimated desigh and fabrication costs for the 
first year of development, and 15%, 10%, and 5% of design and fabrication 
costs for the second, third, and fourth years of development, respectively. 

Tne total estimated engineering change cost for the simulated stations was 

$1.6M. 

Costs associated with procurement of technical data for the simulators 
were estimated to be 7.25% of the estimated design and fabrication costs. 
Estimated technical data costs were $250 thousand (K) dollars. 

The estimated total acquisition cost for the simulated stations is 
therefore $5.2M. 

Simulator Maintenance Cost Estimate 

Estimates of fifteen-year maintenance costs for the simulator were 
developed. It was assumed that major maintenance work on the simulator would 
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be contracted. The manufacturer of the computer specified to drive the 
simulation provided an estimate of labor for one year on the computer and 
associated peripherals. Although it was estimated that the remaining 
electronics in the simulated test station would not equal the computer in 
complexity to maintain, the labor estimate for the computer was doubled to 
arrive at a conservative one-year labor estimate for the entire simulated test 
station. The one-year labor estimate was inflated at 10% per year for the 
fifteen years of the life cycle in order to arrive at the fifteen-year 
estimate of maintenance labor for one simulated test station. The estimate 
for one station was then multiplied by four to arrive at an estimate for all 
four stations. Estimated fifteen-year maintenance labor costs were $432K. 

In order to estimate the costs of spares for the simulation, a percentage 
of the total design and fabrication costs for the four simulated test stations 
was calculated. This percentage varied over the fifteen-year cycle, beginning 
with approximately 9 % of design and fabrication costs per year for the first 
two years, and 1% of design and fabrication costs per year for the remaining 
thirteen years of the life cycle. It was assumed that the majority of 
simulator spares would consist of initial spares lay-in with minimal 
requirements for replenishment spares. This assumption is based on the 
assumed nigh reliability of simulators. Each year's estimates were then 
summed to yield the fifteen-year estimate for spares for all four stations. 

The estimated fifteen-year cost of spares was $722K. 

The total estimate fifteen-year maintenance costs for the simulated AIS is 
therefore $1.15M. 
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Total Cost Estimate for the Simulator 


A summary of the major cost elements for the simulated AIS is provided in 
Table 1. The total fifteen-year cost estimate is $6.35M. 

TABLE 1 

Summary of Major Cost Elements for the Simulated AIS 
Estimated Acquisition Cost $5.2CIM 

Estimated 15-Year Maintenance Cost $1.15M 

Total Estimated Simulated AIS Cost _ $6.35M 

Estimated Actual Equipment Costs 

Data were available from Air Force training device procurement sources on 
the estimated costs for acquisition of all four actual test stations, a 
complete set of thirty-four avionics units and interface test adaptors, 
calibration requirements, and the costs of initial spares for two years for 
the test stations and associated peripheral equipment. Since the analysis of 
station operator functions conducted to specify simulation requirements had 
indicated that only twelve of the thirty-four avionics units and associated 
test adaptors would be required to accomplish the required training, the 
actual equipment costs estimates were reduced 35*. This reduction was 
accomplished in order to make the configuration of actual equipment as 
comparable as possible to that of the simulator. The adjusted total 
acquisition cost for the actual AIS was $7.15M. 

Since the available data provided estimates for the initial two years of 
spares, the remaining thirteen years of spares were calculated at 1* of 
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acquisition costs per year. The estimated fifteen-year cost of snares for the 
actual AIS was $2.25M. 

Data were also available from Air Force sources concerning the estimated 
cost of Air Force manpower required to maintain the actual AIS for fifteen 
years. The total estimated fifteen-year labor cost for the actual AIS was 
$3.3M. 

The total estimated fifteen-year maintenance cost for the actual AIS is 
therefore $5.55M. 

A summary of major cost elements for the actual AIS is orovided in Table 
2. The total fifteen-year cost estimate is $12.7M 

TABLE 2 


Summary of Major Cost Elements for the Actual AIS 


Estimated Acquisition Cost 


$7.]5M 


Estimated 15-Year Maintenance Cost 


$5.55M 


Total Estimated Actual AIS Cost 


$12.7M 


DISCUSSION 

The results indicate that the procurement and maintenance of simulation 
offers a significant potential for cost avoidance when compared with actual 
equipment costs. Total estimated fifteen-year costs for simulated equipment 
were $6.35M, while comparable estimates for actual equipment were $12.7M, a 
savings of 50% with use of simulated equipment. Efforts were made to provide 
a direct comparison of actual and simulated equipment by considering all four 
test stations, specifying equal numbers of avionics units for both actual 
equipment and the simulation, and excluding automated instructional features 
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from consideration. It has been suggested (e.g., Daniels \ Drnnin, 

Daniels, 1976) that simulation permits automation of instructional functions 
tnat can permit significant savings in instructor costs, and, possibly, 
trainee costs. This effort did not address tnis potential cost impact for the 
reasons cited earlier. 

It should be noted that this analysis addressed only the cost aspect of 
choice of training device. The question of the relative training 
effectiveness of simulation and actual equipment *ias beyond the scope of this 
effort. At the present time, there are no reports in tne literature *rtiich 
compare the training effectiveness of simulators and actual equipment for 
intermediate maintenance training. Such training effectiveness data, '«hen 
available, will obviously be of critical importance in choosing a training 
device. 

Based on the present results, plans have been made to initiate the 
detailed LCC analysis. The detailed LCC study will assess the adequacy c' the 
techniques used in this study, with the goal of determining how they might be 
refined for future efforts. Also, the detailed analysis can indicate the 
general level of confidence that can be expected with the methods used to 
develop the estimate in this report. 

Since this study was undertaken at a time when the actual F-16 AIS was in 
a state of flux, some current estimates may require alteration, due to 
decisions made subsequent to the analysis period. Attempts have been made to 
minimize error by using the latest available data. 






The potential value of this effort extends beyond determininq whether or 
not to continue with a subsequent detailed LCC study. It demonstrates a 
method for deriving cost estimates based on limited data. No actual F-16 AIS 
stations were in existence during this work. Further, there were no 
simulators in existence during this work that were of this type of training 
device, although two such simulators were under construction. Therefore, the 
configuration of an appropriate simulation had to be predicted, since there 
were only minimal task analysis data available for the nonexistent AIS 
station. The requirement for LCC estimation in the absence of complete data 
is not atypical, and this report describes methods for providing such 
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